Facial Image Pattern Recognition Based on Triple Space Fusion
GAO Xin-Bo1, WANG Nan-Nan1, PENG Chun-Lei1, LI Cheng-Yuan2
1.State Key Laboratory of Integrated Services Networks, Xidian University, Xi′an 710071 2.Division of Image and Video Surveillance and Forensics, Baoji Public Security Bureau, Baoji 721000
Abstract：With the help of human experience knowledge and cognition, the performance of pattern recognition can be improved for some complex applications, and therefore it is important to construct a pattern recognition system based on the fusion of the physical space, the cyberspace and the cognitive space. In this paper, facial image recognition, which is widely applied in forensic evidence, is taken as an example. Its recent advances in pattern recognition based on triple space fusion are summarized. Sketch based face recognition techniques are introduced from three aspects: synthesis based methods, common space projection based methods and feature descriptor based methods. Some discussions and further development directions are also given. These methods provide technical support for some applications in the field of public security.
[1] Zou W W, Yuen P C. Very Low Resolution Face Recognition Problem. IEEE Trans on Image Processing, 2012, 21(1): 327-340 [2] Zhao W Y, Chellappa R, Phillips P J, et al. Face Recognition: A Literature Survey. ACM Computing Surveys, 2003, 35(4): 399-458 [3] Biswas S, Aggarwal G, Flynn P J, et al. Pose-Robust Recognition of Low-Resolution Face Images. IEEE Trans on Pattern Analysis and Machine Intelligence, 2013, 35(12): 3037-3049 [4] Ren C X, Dai D Q, Yan H. Coupled Kernel Embedding for Low-Resolution Face Image Recognition. IEEE Trans on Image Processing, 2012, 21(8): 3770-3783 [5] Gao X B, Wang N N. Heterogeneous Facial Image Synthesis // Zhang C S, Yang Q, eds. Machine Learning and Its Applications. Beijing, China: Tsinghua University Press, 2013: 79-94 (in Chinese) (高新波,王楠楠.异质人脸图像合成 // 张长水,杨 强,编.机器学习及其应用.北京:清华大学出版社, 2013: 79-94) [6] Wang N N. Research on Heterogeneous Facial Image Synthesis and Its Application. Ph.D Dissertation. Xi′an, China: Xidian University, 2015 (in Chinese) (王楠楠.异质人脸图像合成及其应用研究.博士学位论文.西安:西安电子科技大学, 2015) [7] Wang N N, Tao D C, Gao X B, et al. A Comprehensive Survey to Face Hallucination. International Journal of Computer Vision, 2014, 106(1): 9-30 [8] He X F, Niyogi P. Locality Preserving Projections // Thrun S, Saul L K, Schlkopf B, eds. Advances in Neural Information Processing Systems 16. Cambridge, USA: MIT Press, 2003: 153-160 [9] Roweis S T, Saul L K. Nonlinear Dimensionality Reduction by Locally Linear Embedding. Science, 2000, 290(5500): 2323-2326 [10] Tang X O, Wang X G. Face Photo Recognition Using Sketch // Proc of the International Conference on Image Processing. Rochester, USA, 2006, I: 257-260 [11] Tang X O, Wang X G. Face Sketch Synthesis and Recognition // Proc of the 9th IEEE International Conference on Computer Vision. Nice, France, 2003: 687-694 [12] Tang X O, Wang X G. Face Sketch Recognition. IEEE Trans on Circuit Systems for Video Technology, 2004, 14(1): 50-57 [13] Li Y H, Savvides M, Bhagavatula V. Illumination Tolerant Face Recognition Using a Novel Face from Sketch Synthesis Approach and Advanced Correlation Filters // Proc of the IEEE International Conference on Acoustics, Speech, and Signal Processing. Toulouse, France, 2006, II: 357-360 [14] Turk M A, Pentland A P. Face Recognition Using Eigenfaces // Proc of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Maui, USA, 1991: 586-591 [15] Liu Q S, Tang X O, Jin H L, et al. A Nonlinear Approach for Face Sketch Synthesis and Recognition // Proc of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego, USA, 2005, I: 1005-1010 [16] Gao X B, Wang N N, Tao D C, et al. Face Sketch-Photo Synthesis and Retrieval Using Sparse Representation. IEEE Trans on Circuits Systems for Video Technology, 2012, 22(8): 1213-1226 [17] Wang N N, Li J, Tao D C, et al. Heterogeneous Image Transformation. Pattern Recognition Letters, 2013, 34(1): 77-84 [18] Zhang J W, Wang N N, Gao X B, et al. Face Sketch-Photo Synthesis Based on Support Vector Regression // Proc of the 18th IEEE International Conference on Image Processing. Brussels, Belgium, 2011: 1125-1128 [19] Wang N N, Gao X B, Tao D C, et al. Face Sketch-Photo Synthesis under Multi-dictionary Sparse Representation Framework // Proc of the 6th International Conference on Image and Graphics. Hefei, China, 2011: 82-87 [20] Zhang S C, Gao X B, Wang N N, et al. Face Sketch Synthesis via Sparse Representation-Based Greedy Search. IEEE Trans on Image Processing, 2015, 24(8): 2466-2477 [21] Wright J, Yang A Y, Ganesh A, et al. Robust Face Recognition via Sparse Representation. IEEE Trans on Pattern Analysis and Machine Intelligence, 2009, 31(2): 210-227 [22] Gao X B, Zhang K B, Tao D C, et al. Joint Learning for Single Image Super-Resolution via a Coupled Constraint. IEEE Trans on Image Processing, 2012, 21(2): 469-480 [23] Liu C, Shum H Y, Freeman W T. Face Hallucination: Theory and Practice. International Journal of Computer Vision, 2007, 75(1): 115-134 [24] Vapnik V N. The Nature of Statistical Learning Theory. New York, USA: Springer-Verlag, 1995 [25] Nefian A V, Hayes M. Face Recognition Using an Embedded HMM // Proc of the IEEE Conference on Audio- and Video-Based Biometric Person Authentication. Washington, USA, 1999: 19-24 [26] Rabiner L R. A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. Proceedings of the IEEE. 1989, 77(2): 257-286 [27] Samaria F S. Face Recognition Using Hidden Markov Models. Ph.D Dissertation. Cambridge, UK: University of Cambridge, 1994 [28] Zhong J J, Gao X B, Tian C N. Face Sketch Synthesis Using E-HMM and Selective Ensemble // Proc of the IEEE International Conference on Acoustics, Speech and Signal Processing. Honolulu, USA, 2007, I: 485-488 [29] Gao X B, Zhong J J, Li J, et al. Face Sketch Synthesis Algorithm Based on E-HMM and Selective Ensemble. IEEE Trans on Circuit and Systems for Video Technology, 2008, 18(4): 487-496 [30] Gao X B, Zhong J J, Tao D C, et al. Local Face Sketch Synthesis Learning. Neurocomputing, 2008, 71(10/11/12): 1921-1930 [31] Xiao B, Gao X B, Tao D C, et al. A New Approach for Face Re-cognition by Sketches in Photos. Signal Processing, 2009, 89(8): 1576-1588 [32] Xiao B, Gao X B, Tao D C, et al. Photo-Sketch Synthesis and Recognition Based on Subspace Learning. Neurocomputing, 2010, 73(4/5/6): 840-852 [33] Efros A A, Freeman W T. Image Quilting for Texture Synthesis and Transfer // Proc of the 28th Annual Conference on Computer Graphics. Los Angeles, USA, 2001: 341-346 [34] Wang X G, Tang X O. Face Photo-Sketch Synthesis and Recognition. IEEE Trans on Pattern Analysis and Machine Intelligence, 2009, 31(11): 1955-1967 [35] Pear J. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. 2nd Edition. San Francisco, USA: Morgan Kaufmann, 1988 [36] Yedidia J S, Freeman W T, Weiss Y. Generalized Belief Propagation [EB/OL].[2015-06-28]. http:// papers.nips.cc/paper/1832-generalized-belief-propagation.pdf [37] Zhang W, Wang X G, Tang X O. Lighting and Pose Robust Face Sketch Synthesis // Proc of the 11th European Conference on Computer Vision. Heraklion, Greece, 2010: 420-433 [38] Zhou H, Kuang Z H, Wong K K Y K. Markov Weight Fields for Face Sketch Synthesis // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Providence, USA, 2012: 1091-1097 [39] Wang N N, Tao D C, Gao X B, et al. Transductive Face Sketch-Photo Synthesis. IEEE Trans on Neural Networks and Learning Systems, 2013, 24(9): 1364-1376 [40] Lin D H, Tang X O. Inter-modality Face Recognition // Proc of the 9th European Conference on Computer Vision. Graz, Austria, 2006, IV: 13-26 [41] Yi D, Liu R, Chu R F, et al. Face Matching between Near Infrared and Visible Light Images // Proc of the International Conference on Advances in Biometrics. Seoul, Korea, 2007: 523-530 [42] Lei Z, Li S Z. Coupled Spectral Regression for Matching Heterogeneous Faces // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Miami, USA, 2009: 1123-1128 [43] Yan S C, Xu D, Zhang B Y, et al. Graph Embedding and Extensions: A General Framework for Dimensionality Reduction. IEEE Trans on Pattern Analysis and Machine Intelligence, 2007, 29(1): 40-51 [44] Lei Z, Zhou C T, Yi D, et al. An Improved Coupled Spectral Regression for Heterogeneous Face Recognition // Proc of the 5th IAPR International Conference on Biometrics. New Delhi, India, 2012: 7-12 [45] Sharma A, Jacobs D W. Bypass Synthesis: PLS for Face Recognition with Pose, Low-Resolution and Sketch // Proc of the 24th IEEE Conference on Computer Vision and Pattern Recognition. Colorado Springs, USA, 2011: 593-600 [46] Kan M, Shan S G, Zhang H H, et al. Multi-view Discriminant Analysis // Proc of the 12th European Conference on Computer Vision. Florence, Italy, 2012, I: 808-821 [47] Mignon A, Jurie F. CMML: A New Metric Learning Approach for Cross Modal Matching // Proc of the 11th Asian Conference on Computer Vision. Daejeon, Korea, 2012, I: 1-14 [48] Klare B F, Li Z F, Jain A K. Matching Forensic Sketches to Mug Shot Photos. IEEE Trans on Pattern Analysis and Machine Intelligence, 2011, 33(3): 639-646 [49] Lowe D G. Distinctive Image Features from Scale-Invariant Key-Points. International Journal of Computer Vision, 2004, 60(2): 91-110 [50] Ojala T, Pietikinen M, Menp T. Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns. IEEE Trans on Pattern Analysis and Machine Intelligence, 2002, 24(7): 971-987 [51] Zhang W, Wang X G, Tang X O. Coupled Information-Theoretic Encoding for Face Photo-Sketch Recognition // Proc of the 24th IEEE Conference on Computer Vision and Pattern Recognition. Colorado Springs, USA, 2011: 513-520 [52] Galoogahi H K, Sim T. Face Sketch Recognition by Local Radon Binary Pattern: LRBP // Proc of the 19th IEEE International Conference on Image Processing. Orlando, USA, 2012: 1837-1840 [53] Galoogahi H K, Sim T. Inter-modality Face Sketch Recognition // Proc of the IEEE International Conference on Multimedia and Expo. Melbourne, Australia, 2012, IV: 224-229 [54] Lei Z, Yi D, Li S Z. Discriminant Image Filter Learning for Face Recognition with Local Binary Pattern Like Representation // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Providence, USA, 2012: 2512-2517 [55] Alex A T, Asari V K, Mathew A. Local Difference of Gaussian Binary Pattern: Robust Features for Face Sketch Recognition // Proc of the IEEE International Conference on Systems, Man, and Cybernetics. Manchester, UK, 2013: 1211-1216 [56] Peng C L, Gao X B, Wang N N, et al. Graphical Representation for Heterogeneous Face Recognition [EB/OL]. [2015-06-28]. http://arxiv.org/pdf/1503.00488.pdf
2015, Vol. 28 
